Method and system for testing configuration of environments

ABSTRACT

A method and system for testing configuration of environments are provided. A probe for connection to a network and/or devices interacts to launch a configuration analyser tool. The configuration analyser tool includes a test mechanism for running a set of tests for the probe relating to connectivity and configuration of attached networks and devices before connection and suggesting solutions to test results. The tests may be externalised and dynamically loaded at run time of the configuration analyser tool.

RELATED APPLICATIONS

This application is based on and claims the benefit of priority fromEuropean Patent Application EP 09175366.5, filed Nov. 9, 2009.

BACKGROUND

This invention relates to the field of testing system environments. Inparticular, the invention relates to testing configuration of systemenvironments.

Service level management (SLM) systems are known that collect eventinformation from many different network data sources and present asimplified view of this information to operators and administrators. Forexample, IBM Tivoli Netcool/OMNIbus (IBM, Tivoli, Netcool/OMNIbus aretrade marks of International Business Machines Corporation) is an eventmanagement product which collects enterprise-wide event information, andCA Spectrum (CA Spectrum is a trade mark of CA, Inc.) is a networkinfrastructure management product.

SLMs have object servers to which alert information is forwarded fromintegrations products which are software interfaces between networkmanagers and the endpoints which are being managed. Integrationsproducts are external programs, such as probes, monitors, and gateways.For example, probes connect to an external source, detect and acquireevent data, and forward the data to the object server as alerts.

The alert information is stored and managed at the object server indatabase tables and displayed in an event list.

Integrations product sets such as probes and gateways often give rise toreported problems related to configuration issues. If a customer triesto run an integration product in a mis-configured system environment, itwill fail to launch with little or misleading information regarding whyit failed.

SUMMARY

An embodiment of the invention is a method for testing configuration ofenvironments, including executing a probe for connection to a networkand/or devices, launching a configuration analyser tool from the probe,and running a set of tests for the probe relating to connectivity andconfiguration of attached networks and devices before connection. Theexecuting, launching, and running may be implemented in computerhardware configured to perform the executing, launching, and running, orin computer software embodied in a non-transitory, tangible,computer-readable storage medium.

In one embodiment, the method includes dynamically loading externallystored tests at run time of the configuration analyser tool.

Another embodiment of the invention is a computer program product fortesting configuration of environments, the computer program productincluding a computer readable medium, computer program instructionsoperative to execute a probe for connection to a network and/or devices,launch a configuration analyser tool from the probe, and run a set oftests for the probe relating to connectivity and configuration ofattached networks and devices before connection, where the programinstructions are stored on the computer readable medium.

In one embodiment, the computer program product includes computerprogram instructions operative to dynamically load externally storedtests at run time of the configuration analyser tool.

Another embodiment of the invention is a system for testingconfiguration of environments, including a probe for connection to anetwork and/or devices, a configuration analyser tool launched from theprobe, including a test mechanism for running a set of tests for theprobe relating to connectivity and configuration of attached networksand devices before connection.

In one embodiment, the system includes a test loader component fordynamically loading externally stored tests at run time of theconfiguration analyser tool.

BRIEF DESCRIPTION OF THE DRAWINGS

The subject matter regarded as the invention is particularly pointed outand distinctly claimed in the concluding portion of the specification.The disclosure, both as to organization and method of operation,together with objects, features, and advantages thereof, may best beunderstood by reference to the following detailed description when readwith the accompanying drawings in which:

FIG. 1A is a block diagram of an embodiment of a system in accordancewith the present invention;

FIG. 1B is a block diagram of another embodiment of a system inaccordance with the present invention;

FIG. 2 is a block diagram of a computer system in which an embodiment ofthe invention may be implemented;

FIG. 3 is a flow diagram of an embodiment of a method in accordance withthe present invention; and

FIG. 4 is a schematic diagram of an embodiment of a method of thepresent invention.

DETAILED DESCRIPTION

It will be appreciated that for simplicity and clarity of illustration,elements shown in the figures have not necessarily been drawn to scale.For example, the dimensions of some of the elements may be exaggeratedrelative to other elements for clarity. Further, where consideredappropriate, reference numbers may be repeated among the figures toindicate corresponding or analogous features.

In the following detailed description, numerous specific details are setforth in order to provide a thorough understanding of the disclosure.However, it will be understood by those skilled in the art that thepresent invention may be practiced without these specific details. Inother instances, well-known methods, procedures, and components have notbeen described in detail so as not to obscure the present disclosure.

A method and system are described to provide a flexible solution tomis-configuration problems in integration products. The method andsystem run tests and suggest resolutions for most of the basicconfiguration issues that are encountered by customers on a regularbasis. This allows customers to self-diagnose and fix issues withintheir environment. This also produces detailed output suitable forfeedback to support engineers which can dramatically decrease theturnaround times for issues when they do arise.

The method and system for testing configuration of environments aredescribed in the context of event management integration products.However, the method and system may be used as a generic testingframework in other applications, for example, in automated test harnessframeworks, and automated software fault diagnosis and repair.

Referring to FIG. 1A, a system 100 is shown including a configurationanalyser tool 110A. The configuration analyser tool 110A interfaces witha probe 121.

Probes 121 are external programs which provide alert information forsystems, such as service level management (SLM) systems or eventmanagement systems, that collect event information from many differentnetwork data sources. Probes 121 may include integration probes,monitors, gateways, data collectors, event collectors, etc. Probes 121typically connect to an external source to detect and acquire eventdata, and forward the data to an object server as alerts.

A described probe 121 is provided with an associated diagnosticconfiguration analyser tool 110A that checks for connectivity andconfiguration before connecting both with the event management systemand the network or network device. If there is a problem, then the probe121 will not connect and will wait or hang.

The configuration analyser tool 110A is executed against a probe 121 andis launched by a probe 121 requiring a configuration test. Theconfiguration analyser tool 110A includes a library or jar 111A of probenames 114A and associated test files 112A of tests to be run and asuggestions file 113A of suggested problems for troubleshooting.

The configuration analyser tool 110A includes a test checking mechanism130 which checks the test files' 112A syntax before using the testfiles. The test checking mechanism 130 includes an abort mechanism 131to abort the test if an error is found.

The configuration analyser tool 110A includes a test mechanism 140including establishing a log file 150 for a test. The test mechanism 140includes a test output 142 for outputting the test to the probe 121 anda receiver 143 for the test results from the probe 121.

The test mechanism 140 runs through the test files 112A generating aresult output 144 which outputs suggestions from the suggestions file113A. The result output and suggestions are recorded and stored in thelog file 150.

A user output 145 is provided to a system console 160 with an indicationthat all tests have completed successfully or that a user should checkthe log file 150 for problems with their environment and possiblesolutions.

A system console 160 lists tests which have been run and whether thesewere passed or failed. A user will often refer first to the systemconsole 160 for test results. The log file 150 contains full diagnosticoutput that can be used for further analysis.

FIG. 1B provides an alternative embodiment of a system 100B including aconfiguration analyser tool 110B interfacing with a probe 121.

In the system 100 of FIG. 1B, the tests are externalized from theconfiguration analyser tool 110B and dynamically loaded at run time.This allows the configuration analyser tool 110B to be flexible andcreate a completely modular based toolset. Each individual test istreated as a separate software module, allowing for finer grainedcontrol of the testing environment and dynamic updates of individualtests after the product is shipped.

The logic to run the tests and to interpret the tests may remaininternal to the configuration analyser tool 110B. For example, the logicto run and interpret test XML (Extensible Markup Language), if the testsare this format, remains internal. The test files 112B, for example, inthe form of the test XML and test Binaries, are externalized. The testfiles 112B may be stored in individual versioned JAR (Java Archive)(Java is a trade mark of Sun Microsystems, Inc.) files, allowing forfine grained test version control and ensuring there is no codeduplication.

An external library 111B is provided with test files 112B andsuggestions file 113B associated with probe names 114B. The externallibrary 111B can be stored on a server for access via a network. Thetest mechanism 140 includes a test loader component 141 for loading thetest files 112B and suggestions file 113B at run time of the testmechanism 140.

Referring to FIG. 2, an exemplary system for implementing an embodimentof the configuration analyser tool includes a data processing system 200suitable for storing and/or executing program code including at leastone processor 201 coupled directly or indirectly to memory elementsthrough a bus system 203. The memory elements can include local memoryemployed during actual execution of the program code, bulk storage, andcache memories which provide temporary storage of at least some programcode in order to reduce the number of times code must be retrieved frombulk storage during execution.

The memory elements may include system memory 202 in the form of readonly memory (ROM) 204 and random access memory (RAM) 205. A basicinput/output system (BIOS) 206 may be stored in ROM 204. System software207 may be stored in RAM 205 including operating system software 208.Software applications 210 may also be stored in RAM 205.

The system 200 may also include a primary storage means 211 such as amagnetic hard disk drive and secondary storage means 212 such as amagnetic disc drive and an optical disc drive. The drives and theirassociated computer-readable media provide non-volatile storage ofcomputer-executable instructions, data structures, program modules andother data for the system 200. Software applications may be stored onthe primary and secondary storage means 211, 212 as well as the systemmemory 202.

The computing system 200 may operate in a networked environment usinglogical connections to one or more remote computers via a networkadapter 216.

Input/output devices 213 can be coupled to the system either directly orthrough intervening I/O controllers. A user may enter commands andinformation into the system 200 through input devices such as akeyboard, pointing device, or other input devices (for example,microphone, joy stick, game pad, satellite dish, scanner, or the like).Output devices may include speakers, printers, etc. A display device 214is also connected to system bus 203 via an interface, such as videoadapter 215.

Referring to FIG. 3, a flow diagram 300 illustrates one embodiment of amethod of testing configuration of environments.

A probe is executed 301 which launches 302 a configuration analyser withthe name of the probe. The configuration analyser accesses 303 testfiles to be run and a corresponding suggestions file. In an embodimentin which the test files are externalized, the test files may be loadedat run time.

The test files are syntax tested 304 and it is determined 305 if thereare any errors. If there are errors the testing is aborted 306. If thereare no errors, the method continues.

The test files are executed 307 on the probe sequentially andsuggestions are output from the suggestions file, where required. It isthen determined 308 if the test have completed successfully. If so, asuccessful output message is provided 309. If not, an output message isprovided 311 indicating the log file and/or system console should bechecked for problems and possible solutions.

Referring to FIG. 4, a schematic diagram illustrates the execution of aconfiguration analyser tool against a probe. In one embodiment, aJava-based modular framework is used for the configuration analyser toolto allow a flexible solution.

The probe is executed from the command line 401, with single or multiplearguments, one of them being “-check”. This is symbolically linked 402to a probe wrapper script 403 which will then automatically launch 404the configuration analyser tool 410. The launch of the configurationanalyser tool 410 includes the name of the probe “<probe_name>” as anargument and the probe's “.check.jar” which contains details of tests tobe run as well as an internationalisation file of suggestions in theclasspath.

The configuration analyser tool 410 starts and sets up the log file 450“/logs/<probe_name>.check.log” and then runs 405 “-dumpprops” on theprobe binary 406 and stores the output for use with the tests.

The configuration analyser 410 syntax checks the test file against“/<arch>/ConfigAnalyser.xsd”. If any errors are found it will exit atthis point.

The configuration analyser 410 proceeds to run sequentially through allthe tests 407 detailed in the xml file inside “<probe_name>.check.jar”408, outputting suggestions from the internationalised suggestionsmessage file in that jar.

In an embodiment which includes externalised test files, the processdoes multiple reads and executes on external “Test Library’ modules.

The configuration analyser 410 outputs detailed information 409 to thelog file 450 “/logs/<probe_name>.check.log” and outputs summaryinformation 411 to the system console 460.

Once all tests have been run, the configuration analyser 410 will outputa message either saying all tests completed successfully or one thatsuggests customers check the log file 450 and/or system console 460 forproblems with their environment problems and possible solutions.

The tests verify the connectivity with an event management system andconnectivity with the network before attempting to make a connection.Any problems are documented in a log file.

The tests may include troubleshooting with predictive tests in which anumber of tests are run and a prediction of problems is made.

There are many different tests which may be carried out to test forconfiguration issues. General tests that a configuration analyser toolmay execute include testing if a machine on a network is up, testing ifthere is a firewall between two machines, checking to see if a port on amachine is open, and checking to see if a test file conforms to aspecific regular expression (regex).

An example base set of tests provided in a configuration analyser toolmay include:

-   -   Check if the primary object server is up.    -   Check if the rules and properties file exists.    -   Check if a secondary object server is configured.    -   Check if syntax checker is installed.    -   Check if rules file is syntactically correct.    -   Check if there is space for log files.    -   Check if there is a failover probe configured.

The configuration analyser tool may also contain specific tests for eachintegration product.

For probes, tests can be included to check space for files, check portor socket is available to bind to, and check file syntax, check databaseconnectivity, check Transmission Control Protocol (TCP) serveravailability, check User Datagram Protocol (UDP) port availability, andcheck system library version.

For example, the IBM Tivoli Event Integration Facility (EIF) is an eventdistribution and integration point for an event console. If the probe isan EIF probe, the following tests can be included:

-   -   Check if there is space for EIF cache files.    -   Check if EIF probe can bind to a port.

In another example, if there is a Multi-thread trapd (MTTrapd) probe,the following tests can be included:

-   -   Check gskit and libncrypt library versions.    -   Test auth file syntax.    -   Check socket is available to bind to.

If there is an Open Database Connectivity (ODBC), the following testscan be included:

-   -   Check if ODBC.ini is setup correctly and is referenced by        SODBCINI (On Unix platforms) (Unix is a trade mark of The Open        Group).    -   Check if databases referenced in odbc.props exist.    -   Check if tables and fields referenced in odbc.props exist.

An example test in Extensible Markup Language (XML) is as follows:

  <task classref=“FileExists” name=“Rules file check” description=“Thistest will check to see if the rules file exists”>  <paramname=“fileName” value=“%RulesFile” />  <param name=“passmsg”value=“check.rules_pass”>   <insert value=“%RulesFile” />  </param> <param name=“failmsg” value=“check.rules_fail”>   <insertvalue=“%RulesFile” />  </param> </task>

A “classref” attribute is used to look up the class that needs to beexecuted. “Params” are name value pairs passed to the test itself, withevery test requiring at least a passmsg and failmsg param. Insert valuesare used to dynamically insert environment values(%) or probevalues($)into output strings.

The use of the configuration analyser tool with probes is transparentsince the launch script launches the tool if it finds a “-help” argumentin the probe startup. To the user it looks like it is functionality thatis built into the probe.

A configuration analyser tool and externalised test files may beprovided as a service to a customer over a network.

Embodiments of the invention can take the form of an entirely hardwareembodiment, an entirely software embodiment or an embodiment containingboth hardware and software elements. In one embodiment, the invention isimplemented in software, which includes but is not limited to firmware,resident software, microcode, etc.

Other embodiments of the invention can take the form of a computerprogram product accessible from a computer-usable or computer-readablemedium providing program code for use by or in connection with acomputer or any instruction execution system. For the purposes of thisdescription, a computer usable or computer readable medium can be anyapparatus that can contain, store, communicate, propagate, or transportthe program for use by or in connection with the instruction executionsystem, apparatus or device.

The medium can be an electronic, magnetic, optical, electromagnetic,infrared, or semiconductor system (or apparatus or device) or apropagation medium. Examples of a computer-readable medium include asemiconductor or solid state memory, magnetic tape, a removable computerdiskette, a random access memory (RAM), a read only memory (ROM), arigid magnetic disk and an optical disk. Current examples of opticaldisks include compact disk read only memory (CD-ROM), compact diskread/write (CD-R/W), and DVD.

Improvements and modifications can be made to the foregoing withoutdeparting from the scope of the present invention.

1. A method for testing configuration of environments, comprising:executing a probe for connection to a network; launching a configurationanalyser tool from the probe; dynamically loading one or more externallystored tests at run time of the configuration analyser tool; and runninga set of tests for the probe, the tests relating to connectivity andconfiguration of the network, before connecting to the network.
 2. Themethod as claimed in claim 1, further comprising suggesting solutions toone or more test results.
 3. The method as claimed in claim 1, furthercomprising: storing one or more test results and one or more suggestedsolutions in a log file.
 4. The method as claimed in claim in claim 1,further comprising: providing a summary of tests run and one or moresuggested solutions.
 5. The method as claimed in claim 1, furthercomprising: storing one or more test files for a probe name with anassociated suggestions file; and accessing the stored test files for theprobe in response to launching the configuration analyser tool from theprobe.
 6. The method as claimed in claim 1, further comprising: checkingthe syntax of one or more test files before running the set of tests. 7.The method as claimed in claim 1, wherein if running the set of testsresults in a connectivity or configuration problem, the connection isaborted.
 8. The method as claimed in claim 1, wherein the configurationanalyser tool is transparent to an administrator.
 9. The method asclaimed in claim 1, wherein the method tests event managementintegration products.
 10. The method as claimed in claim 1, wherein theset of tests is selected from the group consisting of testing if aserver is working, testing if a firewall is in place between twomachines, testing to see if a port is open on a machine, testing ifrules and properties files exist, testing if rules and properties filesare syntax correct, testing if a text file conforms to a regularexpression, testing if a syntax checker installed, testing if there isspace for a log file, and testing if a failover probe configured. 11.The method as claimed in claim 1, wherein the probe is an eventmanagement system probe for collecting enterprise-wide eventinformation.
 12. A computer program product for testing configuration ofenvironments, the computer program product comprising: a computerreadable medium; computer program instructions operative to: execute aprobe for connection to a network; launch a configuration analyser toolfrom the probe; dynamically load one or more externally stored tests atrun time of the configuration analyser tool; and run a set of tests forthe probe, the tests relating to connectivity and configuration of thenetwork, before connecting to the network; wherein said programinstructions are stored on said computer readable medium.
 13. Thecomputer program product as claimed in claim 12, further comprisingcomputer program instructions operative to suggest solutions to one ormore test results.
 14. The computer program product as claimed in claim12, further comprising computer program instructions operative to: storeone or more test results and one or more suggested solutions in a logfile.
 15. The computer program product as claimed in claim 12, furthercomprising computer program instructions operative to: provide a summaryof tests run and one or more suggested solutions.
 16. The computerprogram product as claimed in claim 12, further comprising computerprogram instructions operative to: store one or more test files for aprobe name with an associated suggestions file; and access the storedtest files for the probe in response to launching the configurationanalyser tool from the probe.
 17. The computer program product asclaimed in claim 12, further comprising computer program instructionsoperative to: check the syntax of one or more test files before runningthe set of tests.
 18. The computer program product as claimed in claim12, wherein if running the set of tests results in a connectivity orconfiguration problem, the connection is aborted.
 19. The computerprogram product as claimed in claim 12, wherein the configurationanalyser tool is transparent to an administrator.
 20. The computerprogram product as claimed in claim 12, wherein the set of tests isselected from the group consisting of testing if a server is working,testing if a firewall is in place between two machines, testing to seeif a port is open on a machine, testing if rules and properties filesexist, testing if rules and properties files are syntax correct, testingif a text file conforms to a regular expression, testing if a syntaxchecker installed, testing if there is space for a log file, and testingif a failover probe configured.
 21. The computer program product asclaimed in claim 12, wherein the probe is an event management systemprobe for collecting enterprise-wide event information.
 22. A system fortesting configuration of environments, comprising: a probe forconnection to a network; a configuration analyser tool launched from theprobe, including: a test loader component for dynamically loading one ormore externally stored tests at run time of the configuration analysertool; and a test mechanism for running a set of tests for the probe, thetests relating to connectivity and configuration of the network, beforeconnecting to the network.
 23. The system as claimed in claim 22,further comprising: a log file for storing one or more test results andone or more suggested solutions.
 24. The system as claimed in claim 22,further comprising: a library storing test files for a probe name withan associated suggestions file; and the test mechanism accessing thestored test files for the probe launching the configuration analysertool.